Web Survey Bibliography
Relevance & Research Question: Matrix questions are and have always been a big problem in online fieldwork. On the one hand many statistical procedures suggest the use of matrix questions, whilst on the other hand quite a lot of data quality issues are related to matrix questions, - mainly due to satisficing behavior of respondents. One explanation for such a behavior is the high cognitive burden for the respondent when being confronted with the compact presentation of many options and a lot of text on little space. Steps towards a better data quality thus have to find smarter and more intuitive solutions for collecting the needed data. This contribution compares different alternatives for grid questions and shows how to use them in the right context.
Methods & Data: We conducted a study with five splits in total, each of them with a different alternative for the matrix question. Each split contained the respective alternative twice, one time in the context of brand likeability and one time in the context of general attitudes. These concepts have been compared by different metrics in regard to data quality (e.g. variance, time needed on question, consistency with answers in following questions), comparability of the results among these alternatives and the satisfaction of the respondents with usability of each question type. These differences are controlled by sociodemographic factors.
Results: Our results draw a differentiated picture for each of these question types. It seems that there is no universal solution that applies to all contexts. Nonetheless a bunch of recommendations can be drawn for the appropriate design of a questionnaire and the best practice for each of the presented question types.
Added Value: Especially with the growing importance of mobile research in the market research industry, an appropriate questionnaire design gains more and more importance. Smaller screens and new input devices (such as touchscreens) call for alternatives to conventional solutions during fieldwork. We show that researchers most often can count with more satisfied respondents and a better data quality when avoiding the standard matrix question type. Nonetheless they don’t have to do without valuable data.
GOR Homepage (abstract) / (presentation)
Web survey bibliography (4086)
- Is Pretesting Established Among Online Survey Tool Users?; 2012
- An Evaluation of Two Non-Reactive Web Questionnaire Pretesting Methods; 2012; Lenzner, T.
- Recommendations for implementing online surveys and simple experiments in social and behavioural research...; 2012; Hewson, C. M.
- High potential for mobile Web surveys: Findings from a survey representative for German Internet users...; 2012; Funke, F., Wachenfeld, A.
- A taxonomy of paradata for web surveys and computer assisted self interviewing (Casi); 2012; Callegaro, M.
- Can Social Media Research replace traditional research methods?; 2012; Faber, T., Einhorn, M., Hofmann, O., Loeffler, M.
- Bad Boy Matrix Question – Whatcha gonna do when they come for you?; 2012; Tress, F.
- Matrix vs. Single Question Formats in Web Surveys: Results from a large scale experiment; 2012; Klausch, L. T., de Leeuw, E. D., Hox, J., de Jongh, A., Roberts , A.
- Effects of Static versus Dynamic Formatting Instructions for Open-Ended Numerical Questions in Web Surveys...; 2012; Kunz, T., Fuchs, M.
- FamilyVote – Conducting online surveys with children and families; 2012; Geissler, H., Peeters, H.
- The influence of social desirability on data quality in face-to-face and web surveys; 2012; Keusch, F.
- Reducing the Threat of Sensitive Questions in Online Surveys; 2012; Couper, M. P.
- Using the Internet to Administer More Realistic Vignette Experiments; 2012; Caro, F. G., Ho, T., McFadden, D., Gottlieb, A. S., Yee, C., Chan, T.,
- Succinct Survey Measures of Web-Use Skills; 2012; Hargittai, E., Hsieh, Y. P.
- Data Use: Trying to bring the numbers alive ; 2012; Rung, J., Woll, B., Edstrom, C.
- The impact of two-stage highly interesting questions on completion rates and data quality in online...; 2012; Hansen, J. M., Smith, S. M.
- User models as revealed in web-based research services; 2012; Bodoff, D., Raban, D.
- When to Use What Research Design ; 2012; Vogt, W. P., Gardner, D. C., Haeffele, L. M.
- Assessing the Quality of Survey Data ; 2012; Blasius, J.
- Collecting, Managing, and Assessing Data Using Sample Surveys; 2012; Stopher, P.
- Designing and Doing Survey Research; 2012; Andres, L.
- The Impact of Visual Design in Survey Cover Letters on Response and Web Take-Up Rates; 2012; Mockovak, W.
- Validation of an online questionnaire measure of the relative reinforcing value of food; 2012; Reslan, S., Saules, K. K., Greenwald, M. K.
- Monitoring Validated Quality of Life Outcomes After Prostatectomy: Initial Description of Novel Online...; 2012; Sebrow, D., Lavery, H. J., Brajtbord, J. S., Hobbs, A., W., Levinson, A. W., Samadi, D. B.
- Who should supervise students during self-report interviews? A controlled experiment on response behavior...; 2012; Walser, S., Killias, M.
- Web-Based Recruiting for Health Research Using a Social Networking Site: An Exploratory Study; 2012; Fenner, Y., Garland, P., Moore, E. E., Jayasinghe, Y., Fletcher, A., Tabrizi, S. N., Gunasekaran, B.,...
- Exploring Animated Faces Scales in Web Surveys: Drawbacks and Prospects; 2012; Emde, M., Fuchs, M.
- Smart Surveys for Smart Phones: Exploring Various Approaches for Conducting Online Mobile Surveys via...; 2012; Buskirk, T. D., Andrus, C.
- Use of incentives and web-based administration for surveying student alcohol and substance use in an...; 2012; Onoye, J. M., Goebert, D., Nishimura, S.
- Reminders in Web-Based Data Collection: Increasing Response at the Price of Retention?; 2012; Goeritz, A., Crutzen, R.
- Does survey format influence self-disclosure on sensitive question items?; 2012; Kays, K., Gathercoal, K., Buhrow, W.
- Why semantic differentials in Web-based research should be made from visual analogue scales and not...; 2012; Funke, F., Reips, U.-D.
- Using Webinar Polls to Collect Online Survey Data: The Case of a Behavioral Finance Problem; 2012; Sahu, C.
- Non-Response Bias with a Web-Based Survey of College Students: Differences from a Classroom Survey About...; 2012; Wells, W., Cavanaugh, M. R., Nobles, M. R., Bouffard, Je. A.
- The cost-effectiveness of cash versus lottery incentives for a web-based, stated-preference community...; 2012; Gajic, A., Cameron, D., Hurley, J.
- Multiple Sources of Nonobservation Error in Telephone Surveys: Coverage and Nonresponse; 2011; Peytchev, A.; Carley-Baxter, L. R.; Black, M. C.
- Online Questionnaires for Outbreak Investigations; 2011; Parry, A. E.; Johnson, D. R.; Byron-Gray, K.; Raupach, J. C. A.; McPherson, M.
- Inventory of published research: Response burden measurement and reduction in official business statistics...; 2011; Giesen, D. & Snijkers, G. (Eds.), Bavdaz, M., Bergstrom, Y., Gravem, D. F., Haraldsen, G., Hedlin, D...
- Effects of speeding on satisficing in Mixed-Mode Surveys; 2011; Bathelt, S., Bauknecht, J.
- Using Research-Based Practices to Increase Response Rates of Web-Based Surveys; 2011; Perkins, R. A.
- Using break-offs in web interviews for predicting web response in mixed mode surveys; 2011; Beukenhorst, D.
- Web panels in Slovenia; 2011; Lenar, J.
- Traditional and non-traditional treatments for autism spectrum disorder with seizures: an on-line survey...; 2011; Frye, R. E., Sreenivasula, S., Adams, J. B.
- Understanding the new digital divide—A typology of Internet users in Europe; 2011; Brandtzæg, P.B.; Heim, J.; Karahasanoviæ, A.
- Patients’ attitudes toward side effects of antidepressants: an Internet survey; 2011; Kikuchi, T., Uchida, H., Suzuki, T., Watanabe, K., Kashima, H.
- Web-based or paper-based surveys: a quandary?; 2011; Bennett, L., Sid Nair, C.
- Refining the Total Survey Error Perspective; 2011; Smith, T. W.
- ELIPSS: Étude Longitudinale par Internet Pour les Sciences Sociales; 2011; Legleye, S., Lesnard, L.
- Less questions, more data: Revitalizing the european currency in single source affluent audience measurement...; 2011; Hartman, H.
- Linking website exposure data to survey data: A single-source solution; 2011; Krahn, J., Landi, J., Melton, E.